Anatomical recognition, orientation and display of an upper torso to assist breast surgery

ABSTRACT

Various methods, techniques or modules are provided to allow for the automated analysis of the 3-D representation of the upper front torso (i) to recognize 3-D anatomical features, (ii) to orient the subject with reference to their anatomy or a display, (iii) to determine dimensional analysis including direct point-to-point lines, 3-D surface lines, and volume values, (iv) to simulate the outcome with the addition of breast implants including breast and nipple positioning, (v) to assist in the selection of the breast implants, and/or (vi) to assist in the planning of breast surgery. The automated analysis is based on the analysis of changes in a 3-D contour map of the upper torso, orientation analysis of 3-D features and planes, color analysis of 3-D features, and/or dimensional analysis of 3-D features and positions of the upper torso.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.13/644,744, filed Oct. 4, 2012, now issued U.S. Pat. No. 8,648,853,which is a continuation of U.S. patent application Ser. No. 12/319,639,filed Jan. 9, 2009, now U.S. Pat. No. 8,294,708, which claims priorityto U.S. Provisional Application No. 61/010,591, filed on Jan. 9, 2008,the entire disclosure of each of these application being incorporatedherein by reference.

BACKGROUND OF THE INVENTION

In most medical specialties, quantitative diagnostic imaging devices areutilized to assist physicians in diagnosis, operative plan developmentand postoperative analysis, offering accurate measurement andidentification of possible complications. In breast augmentation andreconstructive surgery the main diagnostic tools utilized are the tapemeasure, calipers and camera in tandem with the physicians' “aestheticartistry”. Although successful, reoperative rates for breastaugmentation have remained high.

With breast surgery being one of the most common surgical procedures andwith the introduction of varied types of breast implants, some of whichare in an anatomical form, there is a high likelihood of continued ifnot increased reoperative rates. There would be a benefit to having adiagnostic system that can precisely measure the critical dimensionalparameters to identify possible regions of complications, identificationof patient asymmetries, appropriate sizing and selection of breastimplants and operative plan development, and postoperative analysis anddocumentation. The present invention addresses this need and advancesthe art by providing new techniques for automatic recognition ofanatomical landmarks and features. It is directed to provide consistentand precise measurements between these locations, determination of thebreast volumes and identification of asymmetries, such that they areincorporated in a device to assist breast surgery.

SUMMARY OF THE INVENTION

The present invention provides new techniques for automatic recognitionof anatomical landmarks and features from a three-dimensional (3-D)representation of an upper torso. The various methods, techniques ormodules in this invention allow for the automated analysis of the 3-Drepresentation of the upper front torso (i) to recognize key anatomicalfeatures, (ii) to orient the subject with reference to their anatomy ora display, (iii) to determine dimensional analysis including directpoint-to-point (linear) lines, 3-D surface lines, and volume values,(iv) to simulate the outcome with the addition of breast implants, (v)to assist in the selection of the breast implants, and/or (vi) to assistin the planning of breast surgery. The automated analysis is based onthe analysis of changes in a 3-D contour map of the upper torso,orientation analysis of 3-D features and planes, color analysis of 3-Dfeatures, and/or dimensional analysis of 3-D features and positions ofthe upper torso.

In one embodiment, a computer-implemented method of determining a breastvolume is provided. This is accomplished in an automatic fashion from athree-dimensional (3-D) representation of an upper torso. First a 3-Drepresentation of a breast fold is recognized. The breast fold isdefined as an unclosed curve along the 3-D surface of the 3-Drepresentation defining the lower part of the breast. 3-D chestparameters are recognized from the 3-D representation, which are used todetermine either a partial chest wall pertaining to the recognizedbreast fold or a chest wall including both sides of the chest (alsoreferred to as a full chest wall). Examples of 3-D chest parameters areat least one anterior axillary line corresponding to the side of therecognized breast fold (at least two lines corresponding to the left andright breast fold when determining the volume of both breasts), ananterior chest midline, or a combination thereof.

The partial or full chest wall is optimized to features of a 3-D modelof a chest wall. The breast volume of the original breast shown in the3-D representation is determined from the recognized breast fold, theoptimized partial or full chest wall and the 3-D surface integral of the3-D representation of the upper torso. In case of determining the breastvolume of both breasts (e.g. in one method step) the full chest wallcould be used.

The 3-D representation of the upper torso could be modified by adding a3-D resultant breast to the 3-D surface of either the partial or fullchest wall. The 3-D resultant breast is defined herein as the sum of a3-D breast implant volume and the originally determined breast volumemultiplied by a factor (e.g. ranging from 0.3 to 0.6). The 3-D resultantbreast is positioned near or at the bottom of the corresponding breastfold.

In another embodiment, a computer-implemented method of determiningbreast measurements is provided. Again, this is accomplished in anautomatic fashion from a three-dimensional (3-D) representation of anupper torso. In one example of the automatic recognition of 3-Dfeatures, a contour analysis is performed to the recognized a first setof 3-D features and locations of the upper torso of the 3-Drepresentation. This first set of 3-D features is then used to furtherrecognize a second set of 3-D features and locations of the upper torso.In the recognition of the second of features, a contour analysis, arelational analysis of features, color analysis or dimensional analysiscould be used either individually or in any combination.

Examples of the first set of recognized 3-D features and positionspertain to one or both nipples, one or more anterior axillary lines, theumbilicus, one or more facial features, or the neck, or any combinationthereof. As indicated herein the 3-D representation of the upper torsocould also be in color. Color information is for example useful in therecognition of the areola. Examples of the second set of recognized 3-Dfeatures and positions pertain to the sternal notch, the sternum, one ormore clavicles, one or more chest wall parameters, an upper torsomidline, a coronal plane, one or both areole, one or more breast foldlines, or one or more anterior axillary lines, or any combinationthereof.

Using at least some of the recognized 3-D features and locations, aplurality of 3-D breast-related surface measurements can be determinedin an automatic fashion. For example, one can determine a plurality ofdirect point-to-point breast-related distances or 3-D breast-relatedsurface measurement.

Furthermore, planes and orientations with respect to the upper torso canbe determined in automatic fashion using the recognized 3-D features.For example, the upper torso midline could be determined from at leastone of the clavicles, the sternal notch, one or more of the breast foldlines, the sternum or one or more of the chest wall parameters. Thecoronal plane could be determined from at least one of the clavicles,the sternal notch, the upper torso midline, one or more of the axillarylines, one or more of the breast fold lines, the sternum or one or morechest wall parameters. In other words, the upper torso midline or thecoronal plane of the upper torso could be defined by at least one of thesecond set recognized 3-D features and locations.

One example of a point-to-point breast-related distance is a firstareola diameter in parallel to a transverse plane, whereby thetransverse plane is defined as the plane orthogonal to both the coronalplane and the upper torso midline. Another example of a point-to-pointbreast-related distance is a second areola diameter parallel to themid-sagittal plane, whereby the mid-sagittal plane is defined as theplane orthogonal to both the coronal and transverse planes.

The breast base width is yet another example of a point-to-pointbreast-related distance and corresponds to a breast fold of either theleft or the right breast of the upper torso. In this determination, thebreast fold is automatically recognized from the 3-D representation ofthe upper torso. The breast base width is the projection of the breastfold onto the coronal plane of the upper torso. In one variation, thebreast base width could also be determined at the height of the superiorboundary of the areola of either the left or the right breast of theupper torso. The breast base width is then the projection of a breastfold onto the coronal plane of the upper torso.

Other examples of direct breast related point-to-point distances are,but not limited to, a nipple-to-nipple distance, a nipple-to-midstemalline distance, an intermammary distance, a breast base width, one ormore areola diameters, a mid-clavicle to nipple distance, a breastheight, or a breast-fold line to projected position of the nipple onto achest wall distance, or a combination thereof.

One example of a 3-D breast-related surface measurement is a breast foldcorresponding to either the left or the right breast of the upper torso.The breast fold (also referred to as the breast fold line) is defined asan unclosed curve along the 3-D surface of the 3-D representationdefining the lower part of the breast.

Other examples of 3-D breast-related surface measurement are, but notlimited to, a nipple to breast fold 3-D surface line, a sternal notch tonipple 3-D surface line, a clavicle to nipple 3-D surface line, or amid-clavicle to nipple 3-D surface line, or a combination thereof.

Another measurement that could be automatically determined from the 3-Dmeasurements and 3-D features is a breast cup size. First one wouldrecognize breast fold lines and 3-D nipple features corresponding to theleft and the right breast of the upper torso. Then we determine thelength of a bust curve in a bust plane, whereby the bust plane bisectsthe recognized nipples and is orthogonal to the coronal plane. Anotherlength is determined which is referred to as the length of an inferiorsurface curve. The interior surface curve is defined as a 3-D surfacecurve through a plane inferior of the breast fold lines, whereby theinferior plane is approximately parallel to the transverse plane. Thebreast cup size is based on the difference between the bust curve lengthand the interior surface curve length. This difference could be lookedup in a table that lists the difference in relation to breast cup size.

In yet another embodiment, a computer-implemented method of orientatingand displaying a three-dimensional (3-D) representation of an uppertorso is provided, which is useful for consistency and repeatability ofmeasurements and recognition of 3-D representation(s).

Again, this is accomplished in an automatic fashion from athree-dimensional (3-D) representation of an upper torso. First, aplurality of 3-D features and locations of the upper torso from the 3-Drepresentation of the upper torso. In addition, an upper torso midlineis defined by at least one of the recognized 3-D features and locations.Furthermore, a coronal plane of the upper torso is defined by at leasttwo of the recognized 3-D features and locations. The 3-D representationcan then be rotated and displayed on a display such that the coronalplane of the upper torso coincides with the view plane of the display,and such that the upper torso midline is parallel to the vertical axisof the display.

In one embodiment regarding asymmetry analysis, one or more bisectionlines can be recognized and displayed. Each of the lines bisects ananatomical feature or bisects two anatomical features of the same type(e.g. the nipple of the left or the right breast, the areole of the leftor the right breast, or one or more breast fold lines of the left or theright breast). At least one of the bisection lines can be used as ameasure of asymmetry of the anatomical feature pertaining to the leftand the right breast. A transverse plane can be defined as the planeorthogonal to both the coronal plane and the upper torso midline,whereby the bisection lines can then be displayed parallel to transverseplane.

In another embodiment, a computer-implemented method of visuallycomparing different three-dimensional (3-D) representations of the sameupper torso in an automatic fashion is provided. For a fust 3-Drepresentation, a first set and a second set (as described supra) of 3-Dfeatures and locations of the upper torso is recognized. Then for asecond 3-D representation, a first set and a second set (as describedsupra) of 3-D features and locations of the upper torso is recognized.Given these analyses, the first and the second 3-D representation areorientated towards each other by minimizing one or more of thedifferences in the respective 3-D positions and orientations betweentheir respective first and second set of recognized features in thecorresponding 3-D representations.

In still another embodiment, a computer-implemented method ofdetermining a resultant three-dimensional (3-D) breast shape in anautomatic fashion from a 3-D representation of an upper torso isprovided. The determination of a resultant 3-D breast shape is based onthe recognized breast folds, 3-D chest parameters and the optimizedchest wall (partial or full). With these parameters and features, aresultant 3-D breast shape is added to the 3-D surface of the optimizedchest wall. In this process, the lower boundary of the resultant 3-Dbreast shape is located near or at its respective breast fold line.Furthermore, the resultant breast shape is defined by width, height andprojection parameters. The height is determined by a chest wall heightor a breast fold line, or a combination thereof. The width is less thanor equal to a breast base width. The projection is determined by aresultant breast volume of the resultant 3-D breast. The volume of theresultant 3-D breast shape is defined as the sum of a 3-D breast implantvolume and an originally determined breast volume from the 3-Drepresentation multiplied by a factor (e.g. ranging from 0.3 to 0.6).

In one aspect of this embodiment, the breast fold(s) could be moved inthe inferior, superior, lateral or medial direction along the surface ofthe resultant 3-D breast shape or in any combination of the directions.The resultant 3-D breast shape is then added to the 3-D surface of theoptimized chest wall, whereby the lower boundary of the resultant 3-Dbreast shape is located near or at its respective breast fold line,whereby the breast fold line is in the new and moved position.

In still another embodiment, a resultant 3-D nipple feature and positionon the resultant 3-D breast can be automatically determined from the 3-Drepresentation. In general, the resultant 3-D nipple feature andposition is determined from an originally determined 3-D nipple featureand position in the 3-D representation. Like the breast fold line, theresultant 3-D nipple feature could be moved in the inferior, superior,lateral or medial direction along the surface of the resultant 3-Dbreast shape or in any combination of the directions. In addition, aresultant areola color and 3-D boundary on the resultant 3-D breastcould be automatically determined. In this determination, the 3-D areolaboundary containing the resultant 3-D nipple feature and position.Furthermore, the resultant 3-D areola color boundary could be determinedfrom an originally determined 3-D areola boundary and color in the 3-Drepresentation. Like the resultant breast fold and resultant nipple, theresultant 3-D areola color boundary could be moved in the inferior,superior, lateral or medial direction along the surface of the resultant3-D breast shape or in a combination of the directions.

In still another embodiment, an automatic determination of a resultantnipple medial-to-lateral displacement on the resultant breast shape isprovided. In this example, 3-D features and positions of an originalnipple, an upper torso midline, a coronal plane and a transverse planeare recognized from the 3-D representation. A vector is then determinedperpendicular to the chest wall bisecting the original nipple position,whereby the vector is in the transverse plane.

The resultant nipple medial-to-lateral displacement is defined as theintersection of the resultant breast shape with the vector in thetransverse plane.

In still another embodiment, an automatic determination of a resultantnipple inferior-to-superior displacement on the resultant 3-D breastshape is provided. In this example, 3-D features and position of anoriginal nipple and clavicle corresponding to the side of the nipple, anupper torso midline, a mid-sagittal plane from the 3-D representation.The distance between the nipple and clavicle is determined in a planeparallel to the mid-sagittal plane. The resultant nippleinferior-to-superior displacement is defined by preserving the distancebetween the nipple and the clavicle on the 3-D surface of the resultant3-D breast shape within the plane parallel to the mid-sagittal plane.

The method to determine the resultant nipple superior displacement onthe resultant 3-D breast shape can be varied by analyzing 3-Drepresentations in different postures. For example, a second 3-Drepresentation of an upper torso representing the upper torso with thehands elevated above the head compared to the original or first 3-Drepresentation representing the upper torso with the hands in theproximity of the hips. For both 3-D representations, the breast fold(s),3-D chest parameters, 3-D features of the nipples relative to the chestparameters from their respective 3-D representations are recognized. Theresultant nipple superior displacement is defined as the differencebetween the nipple positions recognized in the (first or original) 3-Drepresentation and the second 3-D representation.

BRIEF DESCRIPTION OF THE FIGURES

The present invention together with its objectives and advantages willbe understood by reading the following description in conjunction withthe drawings, in which:

FIG. 1 shows according to an embodiment of the present invention a 3-Drepresentation of an upper torso 100 (frontal view).

FIG. 2 shows according to an embodiment of the present invention a 3-Drepresentation of an upper torso 200 (front-side view).

FIG. 3 shows according to an embodiment of the present invention a 3-Drepresentation of a neck 300 (frontal view).

FIG. 4 shows according to an embodiment of the present invention a 3-Drepresentation of a face 400 (frontal view).

FIG. 5 shows according to an embodiment of the present invention afrontal view 500 of the upper torso indicating a torso midline 510.

FIG. 6 shows according to an embodiment of the present invention a sideview 600 indicating an axillary line 610.

FIG. 7 shows according to an embodiment of the present invention a 3-Drepresentation of an upper torso in different views (frontal view 710,side view 720 and inferior to superior view 730).

FIG. 8 shows according to an embodiment of the present invention a 3-Drepresentation of a virtual or determined chest wall in different views(frontal view 810, side view 820 and inferior to superior view 830).

FIG. 9 shows according to an embodiment of the present invention afrontal view 900 with chest wall line that are used as chest wallparameters.

FIG. 10 shows according to an embodiment of the present invention viewsin the transverse plane of a breast outline (shown in 1010) that is usedto calculate the length of a bust curve length (shown in 1020) and thelength of an interior surface curve (shown in 1030).

FIG. 11 shows according to an embodiment of the present inventionfrontal plane views to determine asymmetries in, for example, the 3-Dfeatures of the nipples (shown in 1110), areole (shown in 1120) and theinframammary folds (shown in 1130).

FIG. 12 shows according to an alternate embodiment of the presentinvention transverse plane views (1210 and 1220) to determineasymmetries in, for example, the 3-D features of the nipples.

FIG. 13 shows according to an alternate embodiment of the presentinvention transverse plane views (1310 and 1320) to determineasymmetries in, for example, the 3-D features of the nipples.

FIG. 14 shows according to an embodiment of the present invention a 3-Drepresentation of an upper torso in different views with a simulatedresultant breast shape for the left and right breasts (frontal view1410, side view 1420 and inferior to superior view 1430). These figurescan be compared to the original 3-D representation prior to simulationas shown in FIG. 7.

FIG. 15 shows according to an embodiment of the present invention atransverse plane view 1500 of a resultant nipple position withdisplacement in medial to lateral direction.

FIG. 16 shows according to an embodiment of the present invention a sideview 1600 of a overlapping 3-D presentations 720 and 1420 used todetermine a resultant nipple position with displacement in interior tosuperior direction.

FIG. 17 shows according to an alternate embodiment of the presentinvention frontal views 1710, 1720 used to determine a resultant nippleposition with displacement in interior to superior direction.

DETAILED DESCRIPTION

The invention is a computer-implemented method of analyzing in anautomatic fashion a three-dimensional (3-D) representation of an uppertorso. The 3-D representation is a 3-D contour representation of theupper torso, or its associated 4-D contour-color representation, whichis the 3-D contour representation with photographic color applied to it.Herein the 3-D contour representation or the 4-D contour-colorrepresentation is referred to as the 3-D representation.

The 3-D representation can be generated with a variety of techniques andthis invention is independent of the means of generation and/or capturethereof. The 3-D representation of the upper torso of the patient ispreferably obtained with the patient in a specific (and preferablyrepeatable) position. In one example, this position reflects the patientpositioned with their shoulders in a posterior direction, their shoulderblades as close as possible and their arms at their sides. For furtherconsistency their hands may be positioned on their lateral thighs or onhand grips associated with a device. In addition footprints associatedwith the device may be incorporated to preferentially locate the feet.All such aspects of positioning assist in the orientation of the patientwith respect to the device and to provide consistent measurements.

The various methods, techniques or modules in this invention allow forthe automated analysis of the 3-D representation of the upper fronttorso (i) to recognize key anatomical features, (ii) to orient thesubject with reference to their anatomy, (iii) to determine dimensionalanalysis including direct point-to-point (linear) lines, 3-D surfacelines, and volume values, (iv) to simulate the outcome with the additionof breast implants, (v) to assist in the selection of said breastimplants, and/or (vi) to assist in the planning of breast surgery.

Recognition of 3-D Features

The initial step is to perform a 3-D contour analysis on the 3-Drepresentation of the upper torso. In one embodiment, 3-D features andlocations are automatically recognized and analyzed from both thecontours and the rate of change of the contours on the 3-Drepresentation (e.g. 100, 200, 300 and 400 in FIGS. 1-4 respectively).For the purposes of illustration in this application, the examples shownin FIGS. 1-4 are in grey scale where the white regions have no orminimal rate of change, the grey regions have a rate of change reflectedby the grey scale, and the black regions are maxima or minima. As aperson of average skill in the art to which this invention pertainswould readily understand is that color contour maps could also be usedinstead of grey scale contour maps. Examples of anatomical 3-D featuresand locations that can be automatically recognized are one or bothnipples (110, 112), the umbilicus (120), axilla (130, 132), one or moreaxillary lines (also defined as the lateral part of the breast foldline; see 610 in FIG. 6, which also shows elevated arm 620), breast foldlines (140, 142), the sternal notch or head (150), one or more clavicles(160, 162), one or more chest wall parameters (discussed infra), anupper torso midline (170, i.e. the sternum; see also the midsternalline510 in FIG. 5), a coronal plane (discussed infra), one or both areole(180, 182), the sternocleidomastoid muscles (190, 192), the neck (210),trapezius muscles (220, 310), features of the face (e.g. the mouth(410), the corners of the eyes (420, 422), the folds of the sides of thenose (430, 432), the chin (440), or the chin fold (442)). Other featuresnot listed herein could also be recognized from the 3-D representation.Accordingly, the invention is not limited to these recognized features.

In another embodiment, a first set of 3-D features and locations isautomatically recognized and analyzed from both the contours and therate of change of the contours on the 3-D representation. In particular,at least one of the prominent anatomical 3-D features that arerecognized are one or both nipples, the umbilicus, one or more featuresof the face, the neck, one or more breast fold line, one or more(anterior) axillary lines, or any combination thereof.

Once at least some of the first set of 3-D features is recognized, other3-D features (referred to as the second set) can be recognized from the3-D representation using in one example at least one or more of thefirst set of recognized features as a guideline. Examples of the secondset of 3-D features and locations are the sternal notch, one or moreclavicles, one or more chest wall parameters, an upper torso midline(i.e. the sternum), a coronal plane, a sagittal plane, a transverseplane, one or both areole, one or more breast fold lines, or one or moreinterior axillary lines.

Three major (coronal, sagittal and transverse) planes through the uppertorso, and the inferior, superior, medial, lateral, anterior andposterior directions are defined according to anatomical nomenclature,which are automatically recognized. In the 3-D representation, the 3-Dconvex features of the upper torso could be associated with theclavicles. The coronal plane could then be defined by two lines such as:(i) a fitted line through the left and right clavicles, and (ii) theupper torso midline (see also infra). The transverse plane is defined asthe plane orthogonal to both the coronal plane and the upper torsomidline. The remaining sagittal plane is parallel to upper torso midlineand orthogonal to both the coronal and transverse planes.

The orientation aspects of the upper torso can be automatically defined,for example in one exemplary embodiment, by recognizing anatomicalfeatures and comparing their location to a reference, such as the imageillustrated in FIGS. 1-4. For example, if the umbilicus defines afeature that is inferior to most other features it defines the inferiordirection from the center of the 3-D representation. An alternativeapproach is to transform the camera frame of reference 3-Drepresentation to a reference frame determined in the calibration of thesystem or at the point of design, which is not exactly the anatomicalframe of reference. For the purposes of this invention,inferior-superior direction is defined within the coronal plane parallelto the upper torso midline, and has directionality from the umbilicus tonipples to sternal notch and neck. Medial to lateral is defined withinthe coronal plane and is perpendicular to the upper torso midline andhas directionality from midline to nipples to axillae. Posterior toanterior is defined within the transverse plane and is perpendicular tothe upper torso midline and has directionality from sternal notch tonipples.

In general, the rate of change in contours can be calculated from the3-D representation for each point as the difference in x, y, z in nearbycoordinates or their equivalents. Typically for the upper front torsothis yields two prominent local maxima and four prominent local minima.Where usually, these maxima are recognized as the nipples. Theumbilicus, sternal notch and axillae are recognized as the minima. Adetailed set of criteria can be utilized to recognize the 3-D features.

For example, a nipple can be recognized in the 3-D contour rate ofchange analysis as being a local maximum, with a large rate of change inall radial directions with the centers as the maxima over a small regionof about 8 to 15 mm in diameter. For the upper torso, the nipplesusually are the two regions with the greatest rate of change in contoursthat are maxima. The two nipples are recognized and they are separatedby approximately equal to or greater than 15 cm. Other criteria maybeused to further confirm recognition of the nipples, such as confirmingthat they lie within a region that has a darker color than the averageskin color in the 3-D representation.

The umbilicus is recognized in the 3-D contour rate of change analysisas having a local minimum, with a large rate of change in all radialdirections with the centers as the minimum from this center over a smallregion of about an oval or circular region with major and minor axisvarying from approximately 10 to 30 mm, respectively. For the uppertorso, the umbilicus usually is the region with the greatest rate ofchange in contours that is a minimum.

The face has many 3-D features present such as the chin, lips, nose andeyes, which can all be recognized as prominent maxima when looked atwith the contour rate of change analysis (see FIGS. 3-4). Even if onlythe lower part of the face is included in the 3-D representation it isdistinct because of the general surface shape is oval down to the chinand there are many prominent features that are maxima and regions ofconvexity in close proximity to each other. The chin ridge line definingthe jaw line is distinctive. In general, the face can be recognized withthe chin and neck defining the superior part of the torso.

The neck is recognized in the 3-D representation as being the narrowestregion of the torso i.e., shortest distance between two boundaries (seeFIGS. 1-4). An additional criterion that can be used to differentiatethe neck from the surface of the arms and legs is that the neck is inclose proximity to the chin. Moreover, the neck has distinctive 3-Dfeatures when looked at with the contour rate of change analysis. Thechin defines a ridge of convexity (line of maxima). Thesternocleidomastoid muscle defines two prominent convex ridges on theneck's surface defining the anterior triangle of the neck, whichsurrounds a local minima and region of concavity defining the sternalnotch. Lateral to either side of the anterior triangle of the neck arethe triangles concave structures that are also present in the contourrate of change analysis. These are anatomical features for the trapeziusmuscles and which are inferiorly bounded by convex ridges of theclavicles. The sternal notch location could be fine-tuned or refined asthe bisection of this midline and a curve fit to the convex ridgeslateral to the neck associated with the clavicle geometry, as shown inFIGS. 1 and 5. Moreover the sternal notch is the local minimum in aposterior direction closest to the intersections of the midline andclavicle fitted curves.

An individual breast fold line is recognized in the 3-D representation.The breast fold line is the line of transition between the convex breastand the concave regions adjacent to the breast. Breast fold lines couldbe further refined with the knowledge of the nipple locations. Thesearch region could then be restricted to radial distances from thenipple of about 3 to 15 cm rather than the entire upper torso. In oneexemplary embodiment and for the purposes of further analysis in thisinvention, the breast fold line is defined as an unclosed curve alongthe 3-D surface of the 3-D representation defining the lower part of thebreast.

The axillae are determined with the knowledge of additional 3-Dfeatures, such as breast fold lines and the upper torso orientation. Inone example, an axilla could be found as the local minimum superior andlateral to the lateral boundary of the breast fold line. The areolecould be determined using the color boundary information and nipplelocation recognized in the 3-D representation.

The lower torso midline is also visible in the contour rate of changeanalysis, however, it is noted that our analysis derives the upper torsomidline; as such the lower torso midline and position of the umbilicusmay not correspond to the appropriate line. To determine the upper torsomidline, we examine at least two regions: the medial region between thebreast folds superior to the height of the nipples and the medial regionof the neck and more specifically the anterior triangle of the neck, asshown in FIGS. 1-3 and FIG. 5 (indicated by the black dots). In themedial region superior to the height of the nipples between the upperbreast fold lines, data from the fold lines themselves and local contourare reviewed to determine the system plane between left and right sidesof the chest wall. Likewise for medial region of the neck with the lowerleft and right convex features associated with the sternocleidomastoidmuscles. The upper torso midline is determined by fitting the linethrough the middle of these points.

Virtual Chest Wall

This invention includes constructing a virtual chest wall (whereby thesoft tissue envelope may be included or excluded) and subtracting thepatient's 3-D contour surface. A key assumption in this method is thatthe breast tissue (including parenchyma and soft tissue envelope) ispositioned on the anterior chest wall. In one exemplary embodiment, thechest wall under the breasts can be approximately described by thefollowing anatomical features and analyses.

-   -   The breast fold line which determines the boundary of the breast        tissue, as illustrated in FIGS. 1-2. The chest near the breast        folds is approximately a curved line surrounding the breast and        is determined where the distance is about 1 cm radially beyond        the breast fold line;    -   The chest wall's medial curvature in the sagittal plane which is        defined by the sternum curvature along the upper torso midline;    -   The breast fold lines for both sinister and dextro define the        respective lateral lines of curvature of the chest wall (also        referred to as the axillary lines); and    -   To compensate for the presence of the pectoral muscle the        axillary crease horizontal line is obtained from just inferior        of the axilla and bisects the breast fold line to the upper        torso midline. This line also does not have to be a horizontal        line, as long as it connects with the anterior axillary line and        can take a more curved route to the midstemal line.

FIG. 8 shows examples 810, 820, and 830 of the virtual chest wall indifferent view planes, which is defined with these lines of curvatureand can be created utilizing spline interpolation or fitting to afunctional form (e.g. a model of a chest wall, e.g. as shown in FIG. 7)or a combination of both techniques. To improve the anatomical accuracyand continuity between the virtual chest wall with the exposed chestwall, another method for use of spline interpolation is to have morethan one curve (910, 920, 930) enclosing the breast tissue, as shown inFIG. 9. These lines of curvature along the chest wall containinformation relating to the chest wall and are referred to as the chestwall parameters.

Measurements

The feature recognition of the 3-D representation allows one to furtherautomatically calculate breast-related direct point-to-point distances,3-D surface measurements (3-D lines), which are useful to assist inbreast imaging, analyses of breast implants and sizing, analyses ofasymmetries, and breast surgery planning. The following is a descriptionof such measurements and how they are automatically determined(reference for these measurements can be made to FIGS. 1-8).

Point-to-Point Distances

Breast base width. In one example, breast base width corresponding to abreast fold of either the left or the right breast of the upper torso isdetermined as the projection of the breast fold line onto the coronalplane of the upper torso. In another example, breast base width isdetermined at the height of the superior boundary of the areola ofeither the left or the right breast of the upper torso. The breast basewidth is the point-to-point distance between the lateral and medialsides of the projection of a breast fold onto the coronal plane of theupper torso at the appropriate height.

Nipple to Midsternal Line.

The linear measurement of the nipple to midsternal line is determined asthe shortest distance between the 3-D coordinates for the nipples andthe midsternal line.

Areola Diameters.

The recognition of areole is determined using the color boundaryinformation and nipple location recognized in the 3-D representation.The area at which the color darkens from the surrounding tissue is theresulting color boundary of the areola. The areola is typically round inappearance, but our measurements of the 3-D representation determine ageneral round shape. The nipple 3-D feature is contained within theareola boundary, with the nipple generally located at the center of eachareole. Two areola diameter measurements maybe determined for eachareola, one parallel to the transverse plane and the second parallel tothe midsagittal plane, wherein the diameter measurement is apoint-to-point measurement from one edge of the boundary to the otheredge of the boundary of the same areola.

Nipple to Nipple Distance.

The linear measurement between the two nipple positions is calculated asthe shortest distance between the 3-D coordinates of the left and rightnipples.

Intermammary Distance.

The linear measurement of the intermammary distance is calculated as theshortest distance between the 3-D coordinates of the medial boundariesof both breast base fold lines corresponding to the left and rightbreast.

Breast-Fold Line to Projected Position of the Nipple onto a Chest WallDistance.

The linear measurement of the breast fold line to the projected positionof the nipple into the chest wall is calculated as the shortest distancebetween the most inferior point on the breast fold line to theprojection of the nipple onto the virtual chest wall on the left sideand similarly for the right side.

Mid-Clavicle to Nipple Distance.

The linear measurement of the mid-clavicle to nipple distance iscalculated as the shortest distance between the center point of theclavicle on the left side to the left nipple and similarly for the rightside.

Breast Height.

The linear measurement of the breast height is calculated as theshortest distance between the horizontal line that is obtained from justinferior of the axilla and bisects the breast fold line to the midstemalline (this does not have to be a horizontal line as long as it connectswith the anterior axillary line and can take a more curved route to themidstemal line) and the most inferior point of the breast fold line.

Surface Measurements

3-D Sternal Notch to Nipple Distance.

The 3-D surface measurement of the sternal notch to nipple is calculatedas the 3-D line integral over the patient's 3-D contour surface alongthe vector defined between the 3-D coordinates for the sternal notch andthe nipple.

3-D Nipple to Breast Fold Distance.

The 3-D surface measurement of the nipple to inframammary or breast foldis calculated as the 3-D line integral over the patient's 3-D contoursurface along the vector defined as the 3-D coordinates from the nippleto the inframammary breast fold line with the vector being parallel tothe mid-sagittal plane.

3-D Clavicle to Nipple Distance.

The 3-D surface measurement of the clavicle to nipple is calculated asthe 3-D line integral over the patient's 3-D contour surface along thevector parallel to the mid-sagittal plane that bisects the nipple. The3-D distance is determined as the 3-D line integral between the nippleand the clavicle.

3-D Mid-Clavicle to Nipple Distance.

The 3-D surface measurement of mid-clavicle to nipple is calculated asthe 3-D line integral over the patient's 3-D contour surface along thevector defined between the 3-D coordinates for the center point of theclavicle and the nipple.

Breast Cup Size

Breast (also referred to as bra) cup size could be determined in anautomatic fashion from the 3-D representation of the upper front torso.First, the breast fold lines and 3-D nipple features are recognizedcorresponding to the left and the right breasts. A plane is then definedthat contains the nipples and which is orthogonal to the coronal plane.FIG. 10 shows the bust curve which determines the intersection of the3-D representation of the upper front torso bisected by the bust plane,whereby the bust curve starts lateral for the breast fold line andfollows the intersection of the 3-D representation and the plane to theregion of a nipple, at which point the bust curve is linear from onenipple to the other nipple, then follows the intersection of the 3-Drepresentation and the plane to a point which is lateral to the breastfold line. The length of the bust curve is determined (thick line 1040in FIG. 10). A second curve is defined as the inferior surface curve(thick line 1050 in FIG. 10), wherein a 3-D surface line through a planeinferior of the breast fold lines, whereby the plane is approximatelyparallel to the transverse plane or the bust plane. The length of theinferior surface curve is determined, commencing and terminating atpoints along the surface curve (i.e. chest plane 1060) which areparallel to the coronal planes that bisects the commencing point andterminating points on the bust curve. The difference between the bustcurve and inferior surface curve measurement yields a value, which byreferring to saved tables of values yields a bra cup size.

Orientation, Display and Asymmetries

It is noted that difference can be observed in the midlines for theupper and lower torso as well as other regions of the body have alsodifferent midlines. The midline is a key component in deriving theanatomical coordinate system and the corresponding coronal, sagittal andtransverse planes. For breast augmentation the upper torso midline andassociated planes are appropriate. Displaying the torso with respect tothese planes has benefits rather than in the native camera or otherframe of reference.

Informative views of the front, oblique, profile, worm's eye and bird'seye views of the 3-D representation with the torso aligned with themidline in the vertical or horizontal axes can be generated in anautomated fashion.

If the 3-D representation is orientated and displayed such that coronalplane is parallel with the viewing plane and the 3-D representation isfurther orientated such that the midline is parallel to the verticalaxis of the display, asymmetries become apparent. The preferredembodiment for analysis of the asymmetries of left and right anatomicalfeatures of the same type on the 3-D representation are horizontal linesparallel to the transverse plane displayed bisecting the anatomicalfeatures of interest. FIG. 11 exemplifies this for the anatomicalfeatures of the nipple of the left or the right breast, the areola ofthe left or the right breast, or inferior boundary of the breast foldlines of the left or the right breast. The extent of asymmetry can bedetermined by the minimum distance between the bisecting lines in theviewing plane, in this case the coronal plane. FIG. 12 demonstrates anipple projection asymmetry by displaying the 3-D representationorientated with the transverse plane parallel to the viewing plane andthe midsagittal plane parallel to the vertical axis. Two horizontallines (1230) in the coronal plane are displayed each bisectingrespective nipple locations (110, 112). The vertical difference in thehorizontal lines is the degree of asymmetry in projection. In anotherembodiment, the asymmetries can be derived as an angular measurement1240, e.g. the angle relative to the transverse plane, of a line betweenboth nipples, between both areole, or between two breast folds projectedonto the coronal plane (not shown).

FIG. 13 shows another form of displaying asymmetries with respect to thechest wall 1060 with a “worm's eye” view, i.e., the view from inferiorto superior with the viewing plane parallel to the transverse plane. Inthis embodiment the chest parameter line inferior to the breast foldline is displayed and the breast projecting surface is outlined 1330.These two lines maybe reflected about the mid-sagittal plane such thatthe asymmetry of the breast projection and the underlying chest wall canbe observed. This is shown by flipping 1330 from left to right resultingin 1330′ and overlaying 1330 and 1330′ (note that nipples 110, 112 aremirrored for 1330′ compared to 1330). If there is a large change inchest wall with the reflection then, the asymmetry originates in thehard tissue. If there is a large asymmetry in breast projection and notin the chest wall, then the asymmetry originates in the soft tissue.

Breast Volume

Breast volume is determined for an individual breast or for both breastsin a single method step. The technique to determine breast volumerequires recognizing the breast folds defining the lower part of thebreasts from the 3-D representation, the 3-D chest parameters and thevirtual chest wall as discussed supra. The volume of each breast is the3-D integral between the two surfaces. Specifically, the 3-D integral ofthe 3-D representation and the virtual chest wall in the region of thespecific breast described by the chest parameters.

Regarding, the breast volume for an individual breast one requires onlythe associated single breast fold, chest parameters around the singlebreast fold and a virtual chest wall that could either be a partialchest wall or the chest wall including the other breast's 3-D feature.In other words, the volume of the individual breast is then the 3-Dintegral between the 3-D representation and the partial virtual chestwall or virtual chest wall representation with the left and right sideof the chest.

Simulation

The automated recognition of the 3-D features provides a platform toperform simulations of the resultant breast shapes and positions withrespect to the virtual chest well as well as nipple position on theresultant breast shapes. The simulated outcomes can further assist inbreast imaging, analyses of breast implants and sizing, analyses ofasymmetries, and breast surgery planning.

Simulated 3-D forms simulating the outcome with the addition of breastimplants and associated surgery to the existing (also referred to asoriginal—see e.g. FIG. 7 which is the original 3-D representation) formin the 3-D representation creates resultant breasts (FIG. 14). Thecalculated 3-D resultant breast is positioned on the 3-D virtual chestwall (FIG. 8 which is the 3-D determined chest wall of FIG. 7) such thatthe inferior posterior boundary of the resultant breast shape is locatedon or near the breast fold line (e.g. 140, 142). The resultant breasthas a 3-D resultant breast shape that has a height, width andprojection. The volume of each resultant breast shape is equal to theimplant volume plus the volume of the existing breast form with somedegree of atrophy. The degree of atrophy reduces the existing breastvolume by a factor (e.g. in the range of 0.3 to 0.6).

The location of the breast fold lines could be moved in the inferior,superior, lateral or medial direction along the surface of the resultant3-D virtual chest wall or in any combination of these directions. The3-D resultant and simulated breast is updated with the 3-D resultantbreast shape translated so that it corresponds with the translation inthe breast fold line.

In another aspect of the simulation, the resultant 3-D location of thenipple could be determined. In this case, the resultant 3-D simulationis the resultant 3-D breast shape positioned on the virtual chest wall,with the addition of the 3-D nipple feature added at the location of thenipple. The resultant 3-D nipple feature is equal to the original 3-Dnipple feature in the original 3-D representation. Furthermore, the 3-Drepresentation of the areola and the nipple feature on the original 3-Drepresentation could be added to the 3-D simulation with the resultant3-D breast shape at the determined location of the resultant 3-D nipple.Moreover, the color of the areola and nipple from the original 3-Drepresentation could be added to the simulated outcome.

The resultant location of the nipple is dependent on the originallocation of the nipple recognized in the 3-D representation. Like thebreast fold line, the resultant nipple position could also be adjustedin the superior, inferior, lateral, medial, or in any combination ofthese directions.

The resultant 3-D position of the nipple can be determined in a varietyof ways. For example, the displacement of the resultant nipple from theoriginal nipple position could be determined along two directions, i.e.the direction medial-to-lateral and the direction inferior-to-superior.

FIG. 15 shows the resultant nipple medial-to-lateral displacement on theresultant breast shape by determining a vector 1510, 1512 perpendicularto the chest wall 1060 bisecting the original nipple position 110, 112.This vector is in defined in the transverse plane. The resultant nippleposition 1520, 1522 is the point of intersection of the vector with theresultant breast shape.

FIG. 16 shows the resultant nipple inferior-to-superior displacement1610 on the resultant breast shape (1420) by determining the distance1620 between the nipple 112 and the clavicle 162 in a plane parallel tothe mid-sagittal plane on the original 3-D representation 720, whilepreserving the distance between the nipple and the clavicle on the 3-Dsurface of the resultant 3-D breast shape within a plane parallel to themid-sagittal plane (i.e. the two lines 1620 are of constant length).

FIG. 17 shows an alternative method for determining the resultantinferior-to-superior displacement on the resultant breast shape bycomparing the nipple positions of two 3-D representations. In the first3-D representation 1710, the arms are in the proximity of the hips; the3-D nipple features and position are identified relative to the samechest parameters (the first 3-D representation is also referred to asthe 3-D representation herein). In the second 3-D representation 1720,the arms are elevated above the patient's head; the 3-D nipple featuresand position are identified relative to some chest parameters. Theinferior to superior displacement of the nipple position is thendetermined by taking the position difference in corresponding nippleheights (i.e. determined by the difference in lines 1730).

As one of ordinary skill in the art will appreciate, various changes,substitutions, and alterations could be made or otherwise implementedwithout departing from the principles of the present invention. Themethods describes could be programmed in executable computer code andstored on a computer medium or storage medium. The method steps couldalso be codes in various (independent) modules each including one ormore method steps such as for example a measurement module, a featurerecognition module, an asymmetry module, a breast volume module, anorientation module, a simulation module whereby one could furtherdistinguish a breast shape simulation module or a nipple simulationmodule, or the like. The methods or parts of the steps as modules couldalso be integrated in semiconductor or computer chips capable ofperforming or executing the method steps, whereby the chips algorithmscould be called and executed as part of an integrated system. The chipscould for example be used in breast imaging or diagnostic devices orcomputer systems. In other words, the methods steps could be individualmodules or means for executing the methods steps. Accordingly, the scopeof the invention should be determined by the following claims and theirlegal equivalents.

What is claimed is:
 1. A computer implemented method for assistingbreast augmentation surgery planning, the method comprising: generatinga three dimensional representation of an upper front torso of a woman;analyzing a rate of change of at least one contour present in the threedimensional representation of the upper front torso, wherein theanalysis analyzes at least one anatomical feature of the upper fronttorso; constructing a virtual chest wall from the analysis wherein thevirtual chest wall results in a realistic image of the upper front torsoof the woman including an existing breast of the woman; generating fromthe virtual chest wall a three dimensional image of a resultant upperfront torso including a resultant breast of the woman as it would appearafter the breast augmentation surgery, thereby assisting breastaugmentation surgery planning; and determining a volume value of theresultant breast wherein the volume value is equal to the sum of theimplant volume plus the volume of the existing breast multiplied by afactor representing a degree of atrophy of the existing breast volume.2. The method of claim 1, wherein the analyzing a rate of change stepcomprises automatically determining planes and orientations of the upperfront torso of the woman including upper torso midline, coronal plane,sagittal plane, or traverse plane.
 3. The method of claim 1, wherein theat least one anatomical feature includes one or both nipples, one orboth areole, one or more axillary lines, axilla, one or more breast foldlines, sternum, sternal notch, or one or both clavicles.
 4. The methodof claim 1, further comprising determining a breast cup size of theresultant breast.
 5. The method of claim 1, wherein the volume value isequal to the sum of the implant volume plus the volume of the existingbreast multiplied by a factor ranging from 0.3 to 0.6.
 6. The method ofclaim 1, further comprising analyzing an asymmetry of an anatomicalfeature present on the three-dimensional representation.
 7. Acomputer-implemented method for assisting a woman in the planning ofbreast augmentation surgery, the method comprising the steps of:generating a three-dimensional representation of the upper front torsoof the woman; performing a three-dimensional contour analysis byrecognizing and analyzing the contours and the rate of change in thecontours on the three-dimensional representation, wherein thethree-dimensional contour analysis automatically recognizes and analyzeskey anatomical features and locations of the upper front torso of thewoman; constructing a virtual chest wall from the contouredthree-dimensional representation by including a soft tissue envelope,wherein the constructed virtual chest wall results in a realistic imageof the upper front torso of the woman including an existing breast; andgenerating, from the constructed virtual chest wall, a three dimensionalimage of a resultant upper front torso of the woman including aresultant breast, as it would appear after a breast augmentationsurgery, thereby assisting the woman in the planning of the breastaugmentation surgery; and determining a volume value of the resultantbreast wherein the volume value is equal to the sum of the implantvolume plus the volume of the existing breast multiplied by a factorrepresenting a degree of atrophy of the existing breast volume.
 8. Themethod of claim 7, wherein the three-dimensional contour analysisfurther automatically determines planes and orientations of the upperfront torso of the woman including upper torso midline, coronal plane,sagittal plane, or traverse plane.
 9. The method of claim 7, whereinrecognition of key anatomical features and locations include one or bothnipples, one or both areole, one or more axillary lines, axilla, one ormore breast fold lines, sternum, sternal notch, or one or bothclavicles.
 10. The method of claim 7, wherein analysis of at least someof the key anatomical features and locations determines a plurality ofbreast-related direct point-to-point distances and a plurality ofbreast-related three-dimensional surface lines from the recognized keyanatomical features.
 11. The method of claim 7, wherein the plurality ofbreast-related direct point-to-point distances include breast base widthdistance, nipple to midsternal line distance, areola diameter, nipple tonipple distance, intermammary distance, breast fold line to nippleprojection position distance, mid-clavicle to nipple distance, breastheight distance, or any combination thereof.
 12. The method of claim 7,further comprises determining a breast cup size of the resultant breast.13. The method of claim 7, wherein the volume value is equal to the sumof the implant volume plus the volume of the existing breast multipliedby a factor ranging from 0.3 to 0.6.
 14. A computer-implemented methodfor assisting a woman in the planning of breast augmentation surgery,the method comprising: generating a three-dimensional representation ofthe upper front torso of the woman; performing a three-dimensionalcontour analysis by recognizing and analyzing the contours and the rateof change in the contours on the three-dimensional representation,wherein the three-dimensional contour analysis automatically recognizesand analyzes key anatomical features and locations of the upper fronttorso of the woman; constructing a virtual chest wall from the contouredthree-dimensional representation by including a soft tissue envelope,wherein the constructed virtual chest wall results in a realistic imageof the upper front torso of the woman including an existing breast;generating, from the constructed virtual chest wall, a three dimensionalimage of a resultant upper front torso of the woman, including aresultant breast, as it would appear after a breast augmentationsurgery; and determining a cup size of the resultant breast, therebyassisting the woman in the planning of the breast augmentation surgery;determining a volume value of the resultant breast wherein the volumevalue of the resultant breast is equal to the sum of the implant volumeplus the volume of the existing breast multiplied by a factorrepresenting a degree of atrophy of the existing breast volume.
 15. Themethod of claim 14, wherein the volume value is equal to the sum of theimplant volume plus the volume of the existing breast multiplied by afactor ranging from 0.3 to 0.6.